Time of flight sensor module, method, apparatus and computer program for determining distance information based on time of flight sensor data

ABSTRACT

Examples relate to a method, an apparatus and a computer program for determining distance information based on Time of Flight (ToF) sensor data. The method includes obtaining the ToF sensor data, determining one or more saturated regions within the ToF sensor data, determining distance information for one or more boundary regions located adjacent to the one or more saturated regions based on the ToF sensor data, and determining distance information for at least a part of the one or more saturated regions based on the distance information of the one or more boundary regions.

TECHNICAL FIELD

Examples relate to a Time of Flight sensor module and to a method, anapparatus and a computer program for determining distance informationbased on Time of Flight sensor data, more specifically, but notexclusively, to determining distance information for at least a part ofone or more saturated regions based on distance information of boundaryregions located adjacent to the saturated regions.

BACKGROUND

Time-of-flight (ToF) cameras are based on a measurement of a delaybetween the emission of an optical infra-red (IR) signal, which is thenreflected by an object, and the reception of the optical signal at aphoton mixing device (PMD) imager. The measured delay is proportional tothe distance of the object. Not all of the reflections of the infra-redsignal may be usable in the determination of the distance. Portions ofthe infra-red signal that are reflected by a highly reflective object orby an object that is located close to the time-of-flight camera may havean increased amplitude at the PMD imager. If the amplitude of theseportions is too high, then saturated regions may occur within the ToFsensor data.

SUMMARY

Embodiments provide a method for determining distance information basedon ToF sensor data. The method comprises obtaining the ToF sensor data.The method further comprises determining one or more saturated regionswithin the ToF sensor data. The method further comprises determiningdistance information for one or more boundary regions located adjacentto the one or more saturated regions based on the ToF sensor data. Themethod further comprises determining distance information for at least apart of the one or more saturated regions based on the distanceinformation of the one or more boundary regions.

Embodiments provide a computer program product comprising a computerreadable medium having computer readable program code embodied therein,the computer readable program code being configured to implement amethod for determining distance information based on ToF sensor datawhen being loaded on a computer, a processor, or a programmable hardwarecomponent. The method comprises obtaining the ToF sensor data. Themethod further comprises determining one or more saturated regionswithin the ToF sensor data. The method further comprises determiningdistance information for one or more boundary regions located adjacentto the one or more saturated regions based on the ToF sensor data. Themethod further comprises determining distance information for at least apart of the one or more saturated regions based on the distanceinformation of the one or more boundary regions.

Embodiments provide an apparatus for determining distance informationbased on ToF sensor data. The apparatus comprises an interface forobtaining the ToF sensor data. The apparatus comprises a computationmodule configured to determine one or more saturated regions within theToF sensor data. The computation module is configured to determinedistance information for one or more boundary regions located adjacentto the one or more saturated regions based on the ToF sensor data. Thecomputation module is configured to determine distance information forat least a part of the one or more saturated regions based on thedistance information of the one or more boundary regions.

Embodiments provide a Time of Flight (ToF) sensor module comprising aToF sensor and an apparatus for determining distance information basedon Time of Flight sensor data. The apparatus comprises an interface forobtaining the ToF sensor data. The ToF sensor is configured to providethe ToF sensor data. The apparatus comprises a computation moduleconfigured to obtain the ToF sensor data from the ToF sensor. Thecomputation module is configured to determine one or more saturatedregions within the ToF sensor data. The computation module is configuredto determine distance information for one or more boundary regionslocated adjacent to the one or more saturated regions based on the ToFsensor data. The computation module is configured to determine distanceinformation for at least a part of the one or more saturated regionsbased on the distance information of the one or more boundary regions.

BRIEF DESCRIPTION OF THE FIGURES

Some examples of apparatuses and/or methods will be described in thefollowing by way of example only, and with reference to the accompanyingfigures, in which

FIGS. 1 a and 1 b show flow charts of embodiments of a method fordetermining distance information based on Time of Flight sensor data;

FIG. 1 c shows a block diagram of an embodiment of an apparatus fordetermining distance information based on Time of Flight sensor data;

FIGS. 2 a to 2 c show schematic illustrations of measurements taken inan experimental setup; and

FIG. 3 shows an excerpt of a point cloud of a measurement taken in afurther experimental setup.

DETAILED DESCRIPTION

Various examples will now be described more fully with reference to theaccompanying drawings in which some examples are illustrated. In thefigures, the thicknesses of lines, layers and/or regions may beexaggerated for clarity.

Accordingly, while further examples are capable of various modificationsand alternative forms, some particular examples thereof are shown in thefigures and will subsequently be described in detail. However, thisdetailed description does not limit further examples to the particularforms described. Further examples may cover all modifications,equivalents, and alternatives falling within the scope of thedisclosure. Same or like numbers refer to like or similar elementsthroughout the description of the figures, which may be implementedidentically or in modified form when compared to one another whileproviding for the same or a similar functionality.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, the elements may bedirectly connected or coupled or via one or more intervening elements.If two elements A and B are combined using an “or”, this is to beunderstood to disclose all possible combinations, i.e. only A, only B aswell as A and B, if not explicitly or implicitly defined otherwise. Analternative wording for the same combinations is “at least one of A andB” or “A and/or B”. The same applies, mutatis mutandis, for combinationsof more than two Elements.

The terminology used herein for the purpose of describing particularexamples is not intended to be limiting for further examples. Whenever asingular form such as “a,” “an” and “the” is used and using only asingle element is neither explicitly nor implicitly defined as beingmandatory, further examples may also use plural elements to implementthe same functionality. Likewise, when a functionality is subsequentlydescribed as being implemented using multiple elements, further examplesmay implement the same functionality using a single element orprocessing entity. It will be further understood that the terms“comprises,” “comprising,” “includes” and/or “including,” when used,specify the presence of the stated features, integers, steps,operations, processes, acts, elements and/or components, but do notpreclude the presence or addition of one or more other features,integers, steps, operations, processes, acts, elements, componentsand/or any group thereof.

Unless otherwise defined, all terms (including technical and scientificterms) are used herein in their ordinary meaning of the art to which theexamples belong.

At least some embodiments relate to a determination of distanceinformation based on Time of Flight (ToF) sensor data. In Time of Flightdistance measurements, an optical signal (e.g. an infrared signal) maybe emitted, reflected by one or more objects, and measured by aTime-of-Flight sensor comprising a photon mixing device imager. Based ona distance of the object reflecting the optical signal, a delay betweenthe emission of the optical signal and the measurement of the opticalsignal by the ToF sensor may vary. A distance of the object to thesensor may be determined based on the delay between the emission of theoptical signal and the measurement of the optical signal.

In some cases, the reflections caused by an object might not be useablefor distance determination. For example, if the object is too close, orif it comprises a highly reflective surface, such as a reflector, maylead to the ToF sensor data comprising one or more saturated regions,for which a distance determination might not be inherently possible. Atleast some embodiments may circumvent this limitation by detecting theone or more saturated regions within the ToF sensor data. Within the ToFsensor data, boundary regions that are located adjacent to the one ormore saturated regions may be caused by stray light (i.e. scatteredlight) being reflected by the objects that cause the one or moresaturated regions. In the ToF sensor data, this stray light may lead toa corona that surrounds the one or more saturated regions, and which maybe indicative of the distance of the objects to the ToF sensor. Inembodiments, the distance information of the boundary regions may bedetermined, and based on the determined distance information of theboundary regions, which may surround the one or more saturated regions,all or part of the one or more saturated regions may be arbitrarilyattributed distance information that is derived from the distanceinformation determined for the adjacent boundary regions, e.g. based onan interpolation of the distance information of the adjacent boundaryregions. After determining the distance information for the one or moresaturated regions, the distance information for the boundary regions maybe discarded, e.g. declared invalid or unreliable.

FIGS. 1 a and 1 b show flow charts of embodiments of a method fordetermining distance information based on Time of Flight (ToF) sensordata. The method comprises obtaining 110 the ToF sensor data. The methodfurther comprises determining 120 one or more saturated regions withinthe ToF sensor data. The method further comprises determining 140distance information for one or more boundary regions that are locatedadjacent to the one or more saturated regions based on the ToF sensordata. The method further comprises determining 150 distance informationfor at least a part of the one or more saturated regions based on thedistance information of the one or more boundary regions.

FIG. 1 c shows a block diagram of an embodiment of an (corresponding)apparatus 10 for determining distance information based on Time ofFlight (ToF) sensor data. The apparatus 10 comprises an interface 12 forobtaining the ToF sensor data. The apparatus 10 comprises a computationmodule 14 that is coupled with the interface 12. The computation module14 is configured to determine one or more saturated regions within theToF sensor data. The computation module 14 is configured to determinedistance information for one or more boundary regions located adjacentto the one or more saturated regions based on the ToF sensor data. Thecomputation module 14 is configured to determine distance informationfor at least a part of the one or more saturated regions based on thedistance information of the one or more boundary regions. FIG. 1 cfurther shows a Time of Flight sensor module 100, e.g. a ToF camera 100,comprising the apparatus 10 and a ToF sensor 20. The computation module14 may be configured to obtain the ToF sensor data from the ToF sensor20 via the interface 12. The ToF sensor module 100, e.g. the apparatus10 or the computation module 14 of the ToF sensor module may beconfigured to execute the method introduced in connection with FIGS. 1 aand 1 b, e.g. in conjunction with the interface 12.

The following description may relate to the method of FIGS. 1 a or 1 band/or to the apparatus 10 or Time of Flight sensor module 100 of FIG. 1c.

The method is directed on the determination of distance information. Inthis context, distance information may comprise a distance measurementof the distance between a ToF sensor providing the ToF sensor data andone or more objects located in the vicinity of the ToF sensor data. Forexample, the distance information may be represented by a distanceimage, a distance map or by a point cloud. Similar to a “normal” camerasensor, the ToF sensor may provide the ToF sensor data using atwo-dimensional grid of pixels. The ToF sensor data may be representedby a plurality of pixels, e.g. a plurality of pixels arranged in atwo-dimensional grid. Pixels of the two-dimensional grid of pixels mayeach indicate an amplitude/intensity of the light incident to the pixeland distance of the measured object to the ToF sensor that is measuredbased on the incident light. The amplitude/distance image may becalculated by multiple phase measurements with different referencephases. During the determination of the distance information, the phasemeasurements may be an intermediate sensor output that is used tocalculate the distance/intensity. As shown in the following, the phasemeasurements may be used to determine, whether a pixel in the ToF sensordata is saturated. The distance information may represent or indicatethe distance of the measured one or more objects to the ToF sensor. Inat least some examples, the distance information of the one or moresaturated regions within the ToF sensor data may be replaced orsupplanted by the determined distance information of at least a part ofthe one or more saturated regions.

The method comprises obtaining 110 the ToF sensor data. For example, theToF sensor data may be obtained by reading out the ToF sensor data froma ToF sensor (e.g. a PMD imager) providing the ToF sensor data. In atleast some examples, the ToF sensor 20 is configured to provide the ToFsensor data. For example, the ToF sensor data may comprise the amplitudeand the distance information determined by the ToF sensor. In additionor alternatively, the ToF sensor data may comprise the phasemeasurements of the ToF Sensor data. If, for example, the method isexecuted by a ToF sensor module, the ToF sensor data may comprise atleast one phase measurement of light incident to the ToF sensor for eachpixel of the ToF sensor data, and the method may determine the distanceinformation and the amplitude/intensity information based on the atleast one phase measurement of light incident to the ToF sensor. Thephase measurements may be an intermediate sensor output of the ToFsensor during measurement process, which may be utilized to calculatethe distance information and/or the intensity/amplitude information. Insome examples, e.g. if the method is executed in a post-processing ofthe ToF sensor data, the ToF sensor data may comprise the amplitude andthe distance information determined by the ToF sensor, and anindication, whether a pixel of the ToF sensor data is saturated.

The method further comprises determining 120 the one or more saturatedregions within the ToF sensor data. If the ToF sensor data comprisesinformation, whether a pixel within the ToF sensor data is saturated,the determining 120 of the one or more saturated regions may comprisedetecting one or more continuous regions of saturated pixels within theToF sensor data, and determining the one or more saturated regions basedon the one or more continuous regions of saturated pixels. Otherwise,the method may additionally comprise determining which of the pixels aresaturated.

If the method is executed by a ToF sensor module, the method may(additionally) comprise determining, whether a pixel within the ToFsensor data is saturated. In at least some examples, theamplitude/distance image is calculated by multiple phase-measurementswith different reference phases. If a pixel is saturated in at least oneof the phase-measurements, the amplitude/distance information of thepixel may be invalid. Saturated pixels of the single phase-measurementmay be detected by checking whether the digitalized sensor readings arewithin the normal operating range. If the raw value of a pixel isoutside of that range (e.g. close to the minimal or maximal limit), thepixel may be recognized as saturated and might not be used for validamplitude/distance calculation.

In other words, the ToF sensor data may comprise at least one phasemeasurement of light incident to a ToF sensor for each of the pluralityof pixels of the ToF sensor data. The phase measurement may bedetermined relative to at least one reference signal. A pixel of theplurality of pixels may be determined 120 to belong to a saturatedregion of the one or more saturated regions if the phase measurement ofthe pixel is outside a range of valid phase measurements. This mayenable the detection of saturated regions within the raw ToF sensordata. For example, a pixel of the plurality of pixels may be determined120 to belong to a saturated region of the one or more saturated regionsif the phase measurement of the pixel is within a threshold of a maximalor minimal value of the phase measurement. In some examples, the ToFsensor data comprises two or more phase measurements of light incidentto a ToF sensor for each of the plurality of pixels. The two or morephase measurements may be determined relative to two or more referencesignals. A pixel of the plurality of pixels may be determined to belongto a saturated region of the one or more saturated regions if at leastone of the two or more phase measurements of the pixel is outside arange of valid phase measurements. This may increase a reliability ofthe determination.

In at least some examples, as further shown in FIG. 1 b , the methodcomprises identifying 130 the one or more boundary regions within theToF sensor data. The one or more saturated regions may be caused by oneor more objects. The one or more boundary regions may be based on straylight reflected off the one or more objects. In the ToF sensor data, thestray light may form a corona surrounding the one or more saturatedregions. Consequently, the one or more boundary regions may beidentified based to their proximity to the one or more saturatedregions.

In at least some examples, the one or more boundary regions areidentified 130 based on a proximity of the one or more boundary regionsto the one or more saturated regions within the ToF sensor data, e.g.based on a proximity in pixel positions of the one or more boundaryregions to the one or more saturated regions within the ToF sensor data.For example, pixels of the plurality of pixels that are located(directly) adjacent to a saturated pixel within the ToF sensor data (andare not themselves saturated) may be identified to belong to the one ormore boundary regions. Another criterion is the amplitude (i.e.intensity) of the pixels. For example, the ToF sensor data may be basedon a sensing of light incident to a ToF sensor. The one or more boundaryregions may be identified 130 based on an intensity of the sensed lightof the one or more boundary regions within the ToF sensor data. Forexample, the one or more boundary regions may be identified 130 if anintensity of the sensed light of the one or more boundary regions iswithin an intensity range that is indicative of stray light.Alternatively or additionally, the one or more boundary regions may beidentified 130 based on an intensity differential of the sensed lightbetween the one or more boundary regions and adjacent regions within theToF sensor data. For example, an intensity value of the sensed lightwithin the adjacent regions, e.g. further regions within the ToF sensordata, that are not saturated or caused by stray light, may differ bymore than 20% of the intensity value of the sensed light from anintensity value of pixels located within the adjacent regions.

A further point to consider is the extent (i.e. “width”) of the one ormore boundary regions. In a basic example, the extent of the one or moreboundary regions may be arbitrarily set to a given number of pixel(s),such as 1 pixel, i.e. the pixels that are located directly adjacent tothe one or more saturated regions may correspond to the one or moreboundary regions. This may enable a simple determination of the one ormore boundary regions, but may be susceptible to inaccuracies, which maybe smoothed out if the extent of the one or more boundary regions ischosen to be wider. For example, as shown in FIG. 1 b an extent of theone or more boundary regions may be determined 132 based on a distancedifferential and/or based on an intensity differential between the oneor more boundary regions and adjacent regions within the ToF sensordata. For example, if a distance of pixels that are located adjacent topixels comprised in a boundary region of the one or more boundaryregions, is different by at least a threshold value, e.g. by at least 10cm, the extent of the one or more boundary region may be determinedsuch, that the pixels that are located adjacent to the pixels comprisedin the boundary region are excluded from the one or more boundaryregions, e.g. in an iterative or recurrent fashion, starting from thepixels located directly adjacent to the one or more saturated regions.The intensity values of the pixels may be considered similarly. Forexample, if an intensity value of pixels that are located adjacent topixels comprised in a boundary region of the one or more boundaryregions, is different by at least a threshold value, e.g. by at least10% of the intensity value of the pixels comprised in the boundaryregion, the extent of the one or more boundary region may be determinedsuch, that the pixels that are located adjacent to the pixels comprisedin the boundary region are excluded from the one or more boundaryregions. This may enable the determination of the actual extent of theone or more boundary regions and may thus enable a more precisedetermination of the distance information of the one or more boundaryregions.

The method further comprises determining 140 the distance informationfor the one or more boundary regions located adjacent to the one or moresaturated regions based on the ToF sensor data. In some examples, e.g.if the method is executed by a ToF sensor module, the determining 140 ofthe distance information or the one or more boundary regions may bebased on the least one phase measurement of light incident to the ToFsensor for each pixel of the ToF sensor data comprised in the one ormore boundary regions. Alternatively, e.g. if the distance informationof the one or more boundary regions is determined 140 inpost-processing, the determining 140 of the distance information for theone or more boundary regions may comprise extracting the distanceinformation of the one or more boundary regions from the ToF sensordata.

Finally, the method comprises determining 150 distance information forat least a part of the one or more saturated regions based on thedistance information of the one or more boundary regions. As the ToFsensor data might not comprise valid distance information for the one ormore saturated regions, the determining 150 of the distance informationfor at least a part of the one or more saturated regions may correspondto an arbitrary attribution or estimation of the distance informationfor at least a part of the one or more saturated region. In at leastsome examples, the distance information for at least a part of the oneor more saturated regions may be determined (i.e. estimated orarbitrarily attributed) using an interpolation of the distanceinformation of the one or more boundary regions.

In at least some examples, the one or more boundary regions may at leastpartially surround the one or more saturated regions. Distanceinformation of pixels of a boundary region located at opposite ends of asaturated region that is at least partially surrounded by the boundaryregion may be used to interpolate (e.g. using bilinear interpolation)the distance information of at least part of the saturated region. Inother words, the distance information for at least part of the one ormore saturated regions may be determined 150 by interpolating thedistance information for at least part of the one or more saturatedregions based on the distance information of the one or more boundaryregions, e.g. using bilinear interpolation. This may enable anestimation of the distance information for at least part of the one ormore saturated regions. In some examples, a bilinear interpolation maybe used, based on two pairs of pixels located on opposite sides of asaturated region (i.e. pairwise opposite, such that straight lines thatconnect the pixels of the pairs in a two-dimensional representation ofthe ToF sensor data intersect at the pixel of the saturated region, forwhich the distance information is determined). In a computationallyadvanced example, the interpolation may be based on three-dimensionalplane fitting of at least part of the one or more saturated regionsbased on the distance information of the one or more boundary regions.This may enable a more precise determination of the distance informationof the one or more saturated regions.

In some examples, as further shown in FIG. 1 b, the method comprisesdiscarding 160 the distance information of the one or more boundaryregions. This may be desired as the distance information of the one ormore boundary regions may originate from the stray light reflected offthe objects that cause the one or more saturated regions and might notreflect the actual distance at these regions of the ToF sensor data.Instead, the boundary regions may hold distance information for theadjacent saturated regions. They might not represent the distance forthe containing pixel but might hold the right distance values of thesaturated regions at the wrong pixel position. At least some examplesare based on using the out-of-place distance information to determinethe distance information for the saturated regions. In an example, thepoints of the one or more boundary regions may be removed from the pointcloud and/or the distance information of the pixels comprised by the oneor more boundary regions within the ToF sensor data may be marked asinvalid, e.g. by setting a confidence value of the pixels to zero.

The interface 12 may correspond to one or more inputs and/or outputs forreceiving and/or transmitting information, which may be in digital (bit)values according to a specified code, within a module, between modulesor between modules of different entities.

In embodiments, the computation module 14 may be implemented using oneor more processing units or computation units, one or more processingdevices or computation devices, any means for computing, such as aprocessor, a computer or a programmable hardware component beingoperable with accordingly adapted software. In other words, thedescribed function of the computation module 14 may as well beimplemented in software, which is then executed on one or moreprogrammable hardware components. Such hardware components may comprisea general purpose processor, a Digital Signal Processor (DSP), amicro-controller, etc.

The Time of Flight sensor 20 may comprise at least one pixel element,e.g. a photonic mixing device (PMD) or be at least part of an imagesensor circuit, for example, and may include a pixel sensor array, e.g.an array of pixel elements, for example. Each pixel element of the pixelarray may be configured to receive reflected modulated light, which maybe emitted by a time of flight light source and reflected by an object,for example. The ToF sensor 20 may be configured to provide the ToFsensor data comprising the plurality of pixels based on phasemeasurements obtained by the array of pixel elements. In some examples,the ToF sensor module may further comprise a ToF light source configuredto provide the modulated light.

More details and aspects of the method and/or apparatus are mentioned inconnection with the proposed concept or one or more examples describedabove or below (e.g. FIGS. 2 a to 2 c ). The method and/or apparatus maycomprise one or more additional optional features corresponding to oneor more aspects of the proposed concept or one or more examplesdescribed above or below.

At least some embodiments relate to an approximation of saturated areasin time-of-flight images. Saturated pixels in ToF images may causeinvalid distance values. In certain applications (e.g. object detection)it may be crucial to detect objects. In such applications, invalid areasin the distance image may cause these applications to fail. Saturatedpixels are invalid pixels and are often discarded. A saturated pixelstate may be detected, and the corresponding distance values can bemarked as invalid.

These “holes” in the distance image may cause problems since they cannotbe directly assigned with a distance value. The distance of the area inthe hole might be completely unknown and thus may correspond to problemsin certain applications. These holes might also occur with highlyreflective or close objects at the lowest possible exposure time of aToF sensor, e.g. with retroreflectors in signs, street signs, objects,that are too close, or glasses for face recognition.

Since saturated pixels are often caused by too much light arriving atthe sensor, a significant amount of stray-light may occur around theobject reflecting a lot of light. This stray-light may enable validdistance measurements of the saturated area.

FIGS. 2 a to 2 c show schematic illustrations of measurements taken inan experimental setup. In FIG. 2 a , the exemplary experimental setup islaid out. A uniform reflective surface 200, e.g. a piece of whitecardboard, is attached to a pedestal and arranged in a room. In theexperimental setup, the piece of cardboard is arranged too close (forthe used exposure time) to a ToF camera that is used to determinedistance information of the room with the piece of cardboard. FIGS. 2 band 2 c show schematic representations of the amplitude image anddistance image (e.g. of the ToF sensor data) that is recorded of thesetup by a ToF sensor. The piece of cardboard causes a saturated regionwithin the ToF sensor data. In the case of the distance image, a corona(scattered light) 220 is visible around the piece of cardboard 200. Thedistance image shown in FIG. 2 c comprises no valid distance informationfor the piece of cardboard. In the amplitude image illustrated in FIG. 2b , a small corona 210 is also visible around the piece of cardboard. Inan exemplary measurement that was performed based on this setup, theamplitude of the piece of cardboard (the saturated region within the ToFsensor data) was markedly higher than that of the rest of the room, withthe boundary region 210 surrounding the piece of cardboard beingslightly lower (but still markedly higher than that of the rest of theroom).

FIG. 3 shows an excerpt of a point cloud of a measurement taken in afurther experimental setup. In this example, an X-shaped reflector isattached to a box and positioned some way from the ToF sensor. In theamplitude image, the region of the ToF sensor data representing theX-shaped reflector is saturated, while the remainder of the box isbarely visible. As shown in the point cloud of FIG. 3 , in contrast tothe remainder of the box that has the same distance from the ToF sensor,the stray light reflected off the X-shaped reflector yields distanceinformation, which may be used to determine the distance information ofthe saturated region corresponding to the X-shaped reflector.

At least some embodiments may replace the invalid pixel area (e.g. thepiece of cardboard in the above example, or, more general, the one ormore saturated regions) with the interpolated distance values from thesurrounding stray-light. Embodiments may replace the saturated pixelwith the surrounding valid depth data. In some embodiments, thesaturated area may be interpolated by 3D Plane Fitting using thesurrounding stray light pixels as basis.

Especially small objects may be replaced by the interpolation of thecorresponding stray-light.

The aspects and features mentioned and described together with one ormore of the previously detailed examples and figures, may as well becombined with one or more of the other examples in order to replace alike feature of the other example or in order to additionally introducethe feature to the other example.

Examples may further be or relate to a computer program having a programcode for performing one or more of the above methods, when the computerprogram is executed on a computer or processor. Steps, operations orprocesses of various above-described methods may be performed byprogrammed computers or processors. Examples may also cover programstorage devices such as digital data storage media, which are machine,processor or computer readable and encode machine-executable,processor-executable or computer-executable programs of instructions.The instructions perform or cause performing some or all of the acts ofthe above-described methods. The program storage devices may comprise orbe, for instance, digital memories, magnetic storage media such asmagnetic disks and magnetic tapes, hard drives, or optically readabledigital data storage media. Further examples may also cover computers,processors or control units programmed to perform the acts of theabove-described methods or (field) programmable logic arrays ((F)PLAs)or (field) programmable gate arrays ((F)PGAs), programmed to perform theacts of the above-described methods.

The description and drawings merely illustrate the principles of thedisclosure. Furthermore, all examples recited herein are principallyintended expressly to be only for illustrative purposes to aid thereader in understanding the principles of the disclosure and theconcepts contributed by the inventor(s) to furthering the art. Allstatements herein reciting principles, aspects, and examples of thedisclosure, as well as specific examples thereof, are intended toencompass equivalents thereof

A functional block denoted as “means for . . . ” performing a certainfunction may refer to a circuit that is configured to perform a certainfunction. Hence, a “means for s.th.” may be implemented as a “meansconfigured to or suited for s.th.”, such as a device or a circuitconfigured to or suited for the respective task.

Functions of various elements shown in the figures, including anyfunctional blocks labeled as “means”, “means for providing a signal”,“means for generating a signal.”, etc., may be implemented in the formof dedicated hardware, such as “a signal provider”, “a signal processingunit”, “a processor”, “a controller”, etc. as well as hardware capableof executing software in association with appropriate software. Whenprovided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which or all of which may be shared.However, the term “processor” or “controller” is by far not limited tohardware exclusively capable of executing software, but may includedigital signal processor (DSP) hardware, network processor, applicationspecific integrated circuit (ASIC), field programmable gate array(FPGA), read only memory (ROM) for storing software, random accessmemory (RAM), and non-volatile storage. Other hardware, conventionaland/or custom, may also be included.

A block diagram may, for instance, illustrate a high-level circuitdiagram implementing the principles of the disclosure. Similarly, a flowchart, a flow diagram, a state transition diagram, a pseudo code, andthe like may represent various processes, operations or steps, whichmay, for instance, be substantially represented in computer readablemedium and so executed by a computer or processor, whether or not suchcomputer or processor is explicitly shown. Methods disclosed in thespecification or in the claims may be implemented by a device havingmeans for performing each of the respective acts of these methods.

It is to be understood that the disclosure of multiple acts, processes,operations, steps or functions disclosed in the specification or claimsmay not be construed as to be within the specific order, unlessexplicitly or implicitly stated otherwise, for instance for technicalreasons. Therefore, the disclosure of multiple acts or functions willnot limit these to a particular order unless such acts or functions arenot interchangeable for technical reasons. Furthermore, in some examplesa single act, function, process, operation or step may include or may bebroken into multiple sub-acts, -functions, -processes, -operations or-steps, respectively. Such sub acts may be included and part of thedisclosure of this single act unless explicitly excluded.

Furthermore, the following claims are hereby incorporated into thedetailed description, where each claim may stand on its own as aseparate example. While each claim may stand on its own as a separateexample, it is to be noted that—although a dependent claim may refer inthe claims to a specific combination with one or more other claims—otherexamples may also include a combination of the dependent claim with thesubject matter of each other dependent or independent claim. Suchcombinations are explicitly proposed herein unless it is stated that aspecific combination is not intended. Furthermore, it is intended toinclude also features of a claim to any other independent claim even ifthis claim is not directly made dependent to the independent claim.

What is claimed is:
 1. A method for determining distance informationbased on Time of Flight (ToF) sensor data, the method comprising:obtaining the ToF sensor data, wherein the ToF sensor data is based on asensing of light incident to a ToF sensor; determining one or moresaturated regions within the ToF sensor data; identifying one or moreboundary regions located adjacent to the one or more saturated regionswithin the ToF sensor data, based on an intensity of the sensed light ofthe one or more boundary regions within the ToF sensor data; determiningan extent of the one or more boundary regions based on an intensitydifferential between the one or more boundary regions and adjacentregions within the ToF sensor data; determining distance information forthe one or more boundary regions based on the ToF sensor data and takinginto account the extent of the one or more boundary regions; anddetermining distance information for at least a part of the one or moresaturated regions based on the distance information of the one or moreboundary regions.
 2. The method of claim 1, wherein determining theextent of the one or more boundary regions is based on comparing anintensity differential between the one or more boundary regions andadjacent regions within the ToF sensor data to a threshold value.
 3. Themethod of claim 2, wherein the method comprises determining the extentof one or more boundary regions so as to exclude, from the boundaryregion, pixels adjacent to the one or more boundary regions havingintensity values or distance values that are different, by more than athreshold value, from neighboring pixels in the one or more boundaryregions.
 4. The method of claim 1, wherein the distance information forat least part of the one or more saturated regions is determined byinterpolating the distance information for at least part of the one ormore saturated regions based on the distance information of the one ormore boundary regions.
 5. The method of claim 4, wherein theinterpolation is based on three-dimensional plane fitting of at leastpart of the one or more saturated regions based on the distanceinformation of the one or more boundary regions.
 6. The method of claim1, further comprising removing points corresponding to the one or moreboundary regions from a point cloud.
 7. The method of claim 1, whereinthe one or more saturated regions are caused by one or more objects, andwherein the one or more boundary regions are based on stray lightreflected off the one or more objects.
 8. The method of claim 1, whereinthe ToF sensor data is represented by a plurality of pixels, wherein theToF sensor data comprises at least one phase measurement of lightincident to a ToF sensor for each of the plurality of pixels, whereinthe phase measurement is determined relative to at least one referencesignal, and wherein a pixel of the plurality of pixels is determined tobelong to a saturated region of the one or more saturated regions if thephase measurement of the pixel is outside a range of valid phasemeasurements.
 9. The method of claim 8, wherein the ToF sensor datacomprises two or more phase measurements of light incident to a ToFsensor for each of the plurality of pixels, wherein two or more phasemeasurements are determined relative to two or more reference signals,and wherein a pixel of the plurality of pixels is determined to belongto a saturated region of the one or more saturated regions if at leastone of the two or more phase measurements of the pixel is outside arange of valid phase measurements.
 10. A non-transitory computerreadable medium having computer readable program code embodied therein,the computer readable program code being configured to implement themethod of claim 1, when being loaded on a computer, a processor, or aprogrammable hardware component.
 11. An apparatus for determiningdistance information based on Time of Flight (ToF) sensor data, theapparatus comprising: an interface circuit configured to obtain the ToFsensor data, wherein the ToF sensor data is based on a sensing of lightincident to a ToF sensor; and a processing circuit configured to:determine one or more saturated regions within the ToF sensor data;identify one or more boundary regions located adjacent to the one ormore saturated regions within the ToF sensor data, based on an intensityof the sensed light of the one or more boundary regions within the ToFsensor data; determine an extent of the one or more boundary regionsbased on an intensity differential between the one or more boundaryregions and adjacent regions within the ToF sensor data; determinedistance information for the one or more boundary regions based on theToF sensor data and taking into account the extent of the one or moreboundary regions; and determine distance information for at least a partof the one or more saturated regions based on the distance informationof the one or more boundary regions.
 12. A Time-of-Flight (ToF) sensormodule comprising a ToF sensor and the apparatus of claim 11, whereinthe ToF sensor is configured to provide the ToF sensor data, and whereinthe computation module processing circuit is configured to obtain theToF sensor data from the ToF sensor via the interface circuit.
 13. Theapparatus of claim 11, wherein the processing circuit is configured todetermine the extent of the one or more boundary regions based oncomparing an intensity differential between the one or more boundaryregions and adjacent regions within the ToF sensor data to a thresholdvalue.
 14. The apparatus of claim 13, wherein the processing circuit isconfigured to determine the extent of one or more boundary regions so asto exclude, from the boundary region, pixels adjacent to the one or moreboundary regions having intensity values or distance values that aredifferent, by more than a threshold value, from neighboring pixels inthe one or more boundary regions.
 15. A method for determining distanceinformation based on Time of Flight (ToF) sensor data, the methodcomprising: obtaining the ToF sensor data, wherein the ToF sensor datais based on a sensing of light incident to a ToF sensor; determining oneor more saturated regions within the ToF sensor data; identifying one ormore boundary regions located adjacent to the one or more saturatedregions within the ToF sensor data, based on an intensity of the sensedlight of the one or more boundary regions within the ToF sensor data;determining a width of each of the one or more boundary regions based onan intensity differential between the one or more boundary regions andadjacent regions within the ToF sensor data; determining distanceinformation for the one or more boundary regions based on the ToF sensordata and taking into account the width of the one or more boundaryregions; and determining distance information for at least a part of theone or more saturated regions based on the distance information of theone or more boundary regions.
 16. The method of claim 15, whereindetermining the width of the one or more boundary regions is based oncomparing an intensity differential between the one or more boundaryregions and adjacent regions within the ToF sensor data to a thresholdvalue.
 17. The method of claim 15, wherein the distance information forat least part of the one or more saturated regions is determined byinterpolating the distance information for at least part of the one ormore saturated regions based on the distance information of the one ormore boundary regions.
 18. The method of claim 17, wherein theinterpolation is based on three-dimensional plane fitting of at leastpart of the one or more saturated regions based on the distanceinformation of the one or more boundary regions.
 19. The method of claim15, further comprising removing points corresponding to the one or moreboundary regions from a point cloud.
 20. The method of claim 15, whereinthe ToF sensor data is represented by a plurality of pixels, wherein theToF sensor data comprises at least one phase measurement of lightincident to a ToF sensor for each of the plurality of pixels, whereinthe phase measurement is determined relative to at least one referencesignal, and wherein a pixel of the plurality of pixels is determined tobelong to a saturated region of the one or more saturated regions if thephase measurement of the pixel is outside a range of valid phasemeasurements.